In this video, we will
cover the basics of morphology and lexicography
in more detail. Remember that morphology is
the study of the structure of words, while lexicography is a study of
identifying words and phrases. Let's start with morphology. Morphology breaks words
down into their stems or root words, prefixes and suffixes. A stem is the core part of the word, while prefixes are added to
the beginning of the stem and suffixes are added
at the end of the stems. There are three main types of ways to connect
these word components, inflection, derivation,
and compounding. Inflection takes a stem word
and makes them plural, past tense, or progressive. For example, for
the stem word 'like', inflection connections
include the plural likes, the past tense liked, and the progressive liking. Derivation combines the word
stem with syntactic morphemes. For example, combining the stem
print with er forms printer. Or combining print with
able forms printable. You can also combine with
prefixes like re to form reprint. Finally, compounding is combining two root words together
to form a new word. For example, you can combine house
and boat to create houseboat. But the order matters as
you could also combine boat and house to create boat house. We use morphology and
NLP in a few ways. Stemming is the practice of identifying the stem
or root of the word. This is a useful technique
when searching notes, because searching 'lik'
will return like, likes, liked and liking. We can also use
the inflection combinations to understand the part of speech when applying
the syntactic approach for part of speech tagging. NLP also requires the concepts
inherent in lexicography. The first lexicography challenge is to actually recognize the word. This process is called tokenization, a technique for identifying
the word boundary. You can use regular expressions
to identify white space. Things like spaces and tabs or even punctuation to
separate outwards. However, this isn't always so
easy in the biomedical domain. For example, sometimes
multi-word phrases actually have a single meaning. The disease Charcot-Marie-Tooth
disorder is one example. BID is an abbreviation
that means twice a day. Other biological names also
have this multi-word structure, things like H. pylori. Once you get past
the challenge of tokenization, then you actually have
to identify the words. The simplest approach is
a Brute Force Lexical Lookup. Essentially search a dictionary
of all possible words. But creating that dictionary
is difficult because every single word would need to have all the inflection and
derivation combinations. The other option is to break
up the tokens into root words and reverse the
morphologic combinations which can be very complex. Some of the challenges of
real-world NLP applications are just coming up with the appropriate lexicons for
the text you're working with. Because health care has so many
proper names and abbreviations, this process is very challenging.